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(54) Abstract Title 

Virtual studio projection system 

(57) In a virtual studio the actor 18 on the foreground area 14 is provided by a projector 30 with a projected 
plan of each 3D virtual object to be placed in the foreground area to enable the actor to move within the 
foreground area without "colliding" with the virtual objects. The projector 30 may be synchronised with a 
television camera 16 so that the projector 30 is on when the camera 16 is off. 
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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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VIRTUAL STUDIO PROMOTION SYSTEM 



The present invention relates to a virtual studio projection system 
and more particularly to a system for providing actors in a virtual studio 
5 with an indication of the presence of virtual objects. 

In a normal television studio or set there are usually provided 
several 3D foreground objects around which actors can move. The extent 
of such objects can be seen by the actors who are therefore able to move 
10 round the foreground objects. The background is usually provided by one 
or more suitably painted backdrops. 

In an intermediate arrangement of a virtual set the background can 
be created and adjusted to camera view point by using a chroma-key panel 

15 or sensors to measure the orientation of the camera and focal length of the 
lens and a suitable background scene can be viewed with foreground 
objects superimposed on, for example, a video image. In this case real 
3D foreground objects such as desks can be positioned in the foreground 
area and the actor or presenter can be seated at such a desk and can move 

20 in front of the desk. 

However, if a virtual 3D object is "positioned" in the foreground 
area the actor cannot see it and in order to move round it the actor must 
estimate where it is. This is usually done by rehearsal. 

25 

If the virtual object is say, for example, a pillar or other stationary 
object, then the actor can learn its position. In some circumstances the 
position can be marked on the floor by a small mark which is small 
enough not to be too visible by the studio cameras. 

30 
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This may be acceptable if the object is small and of a regular shape, 
in which case the centre can be marked by a small cross but if the object 
is large or irregular in shape then the actor will not be able to move round 
it with any confidence. If several actors are involved in a scene then 
mistakes are usually made and a retake wall be necessary if, for example, 
an actor walks "through" a "solid" virtual object. The situation is farther 
compounded if the virtual object is able to move, as is the case of an 
animated character or a moving object. 

It is an objective of the present invention to provide a means for 
enabling an actor to know where a virtual 3D object is. Preferably the 
actor will be provided with an outline of the virtual object continuously 
throughout a scene. 

The present invention therefore provides a virtual studio projection 
system comprising projector means for projecting on to the foreground 
area floor of a virtual studio a projection plan profile of virtual 3D 
objects. 

In a preferred embodiment the projection means comprises a video 
projector situated above the foreground floor area. In a further 
embodiment one or more such video projectors may be used to cover a 
large area. 

The foreground area may preferably be a chroma-key coloured 
surface, for example blue. 

The video projector may project on to the floor an outline in black 
and white which will appear to the actors eyes as dark/bright blue if the 
floor area is painted in plain blue. In this case the actor will see a 



contrast image of the projection plan outline of the 3D objects. 

In accordance with a further embodiment, the TV camera and the 
video projector are synchronised such that they are not operative together. 

5 

The TV camera is, for example, arranged to be operative for three 
quarters (75%) of a video field by adjusting its electronic shutter and the 
overhead projector camera for the other quarter (25%). Preferably, in a 
practical system, there will be a time delay between switching to ensure 
10 no overlap. 

Preferably the synchronisation is done by adjusting the electronic 
shutter of the camera. 

15 Thus, the actor can see the outlines continuously because they will 

appear 50 times each second. The actor may observe a slight flickering 
but this should not be intrusive. The outline will be clear to all actors. 

If there are several actors and several 3D objects then the use of 
20 multiple synchronised video projectors will be beneficial ensuring adequate 
imaging of the projected plan views even if an actor blocks one of the 
projector* Also if the stage foreground area is large, it may be desirable 
to use several video projectors. 

25 In certain cases objects are not vertically straight. For example, a 

vase may be curved and an archway will have a curved top. In a 
preferred embodiment the size of the virtual object at the height of each 
actor is sampled and this is the projected plan. Thus, for an arch only the 
support pillars footprints would normally be projected. 

30 
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Embodiments of the present invention will now be described, by 
way of example with reference to the accompanying drawings in which : 

Figure 1 shows schematically in side elevation a virtual studio 
5 system illustrating the position of a TV camera and a video projector; 

Figure 2 shows schematically in front elevation a virtual studio 
system showing two video projectors; 

Figure 3 shows in block diagrammatic form control circuitry for the 
arrangement of Figure 2; 
[0 Figure 4 illustrates in a first embodiment a method for generating 

the floor plan of a virtual object; 

Figure 5 illustrates in a second embodiment a method for generating 
the floor plan of a virtual object; 

Figure 6 illustrates the generation of an orthogonal floor plan; 
15 Figure 7 shows block diagrammatically the system of the present 

invention with 3D animated models; and 

Figure 8 is a flow diagram illustrating the operation of the system 
according to the present invention. 

20 With reference now to Figure 1, a virtual studio 10 is shown 

schematically and comprises a chroma-key background panel 12 and a 
foreground floor area 14 which may also be a chroma-key colour. A TV 
camera 16 is used to photograph foreground objects 18 and electronic 
processing circuitry 20 is used to incorporate each foreground object 18 

25 into a virtual background adjusted to camera viewpoint. The chroma-key 
panel may be of the type disclosed in PCT Patent Applications Nos. 
US95/05324 and GB96/02227 to the present applicant, the descriptions of 
which are incorporated herein for technical explanation. The invention 
can also be used for sensor based virtual sets. 
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In the present invention a video projector 30 is mounted in a 
suitable position above the foreground floor area 14. The video projector 
30 is provided with a lens 302 which enables it to project images over the 
floor area 14 as shown by dotted lines 304,306. 

5 

The video projector 30 is preferably linked to the studio electronic 
apparatus 20 to be able to be synchronised with the gate sync of the 
camera 16. 

10 If required, as shown in Figure 2, more than one projector 30,32 

may be used. This is beneficial to ensure approximately equal lighting for 
a large floor area which will be covered as shown by dotted lines 
304,306,308,310. The two video projectors will be synchronised as 
indicated in Figure 3. 

15 

In Figure 3 the first and second projectors 30,32 are connected to 
a floor plan rendering machine 40 which in a preferred embodiment is 
synchronised to the studio apparatus sync 42 from apparatus 20. 

20 The floor plan rendering machine receives signals from a 3D 

graphics computer 44 which may, for example, be a Silicon Graphics 0 2 
model. 



The output of the floor plan rendering machine will be explained 
25 with reference to Figures 4 to 8 but essentially comprises a pattern 
showing the orthogonal projected outline of all 3D virtual objects which 
the computer 44 is programmed to place into the area 14. 

The projection can be a black/white pattern- If the floor area is 
30 painted blue then the black/white image will show up as dark/bright blue 
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on the floor giving the actor a shaded pattern. If the chroma-keyer has a 
frequency bandwidth which will accept both shades (see above referenced 
co-pending applications) then neither shade of blue will show on the 
combined video output. However, such shades may be difficult for the 
5 actors to see especially if the 3D object is moving and the studio lighting 
is very bright. The actor may have to look carefully and this may detract 
from the natural movements/expression of the actor. 

In the preferred embodiment therefore the pattern is a black and 
10 white pattern and the video image projector is a black and white projector. 
In a preferred example the white projection areas which appear as light 
blue are where the actors may move, the black areas which appear as dark 
blue representing the 3D objects. 

15 Referring now to Figures 4 to 6, the floor plan rendering machine 

40 generates a pattern which comprises for each 3D virtual object to be 
inserted into the area 14 an orthogonal plan view of the object at a height 
H. the height H can be different for each actor. 

20 For the purposes of explanation, two example are given but firstly 

by reference to figure 6 the orthogonal plan outline is explained. The 
outline is i©t a perspective "view" of the object and is shown in dotted 
lines 324,326. This is generated as shown in figures 4 and 5 in which F 
represents the correct floor plan for an actor of height H and F 1 for an 

25 actor (not shown) of height H 1 . As can be seen in Figure 4, the plan F 
represents the bulged part B of the urn 328, this being the largest 
orthogonal plan covered by the urn for an actor of height H. Thus, all 
areas outside F will be white and the are F (in this case a circle will be 
black). 

30 
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If, however, the actor is taller (or the urn is smaller) the relative 
height will be H 1 and in this case the actor will have to avoid area F l to 
not collide with the crown or rim C of the urn. 

5 In Figure 5 the importance of this can be seen clearly for an 

archway 340. the actor can walk through the centre 342 of the archway 
which will be outlined as white but not through the support pillars shown 
as F. If the archway has side walls 344,346 then these will also extend 
the region F longitudinally thereby clearly showing the actor the opening 

10 in the archway. 

With reference now to Figure 7, it is possible also to introduce 3D 
animated characters using an animation controller 70. 

15 In operation title animation controller 70 controls the movements of 

a 3D character. The character can be moved in accordance with a set 
computer program or can be moved using sensors attached to a human 
model. 

20 The 3D character is in known manner displayed as a foreground 

object by a graphics computer 50. The character can be generated or 
stored as a 3D model in a store 60. 

The 3D character will change in size and shape and this will be 
25 computed by the graphics computer 50. The computer can then be used 
to produce the necessary instruction to the floorplan rendering machine 
to ensure that the projected plan view of the 3D animated (or moving) 
character appears in the correct position and shape on the floor area 14. 

30 An exemplary flow diagram is shown in Figure 8. 
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Following start 800 and initial set 802, the basic floor pattern will 
be projected on to the floor. 



With reference to Figure 7, if the pattern is in black and white a 
5 studio sync pulse 202 will be provided to the floor plan rendering machine 
40. This sync pulse is used to control the ON/OFF times of the video 
projector 30 (and 32 etc if present). The video projector is switched on 
a maximum time period not longer man the period during which exposure 
of the TV camera 16 is off. Thus, if TV camera 16 is exposed for 75 % 
10 of a field period then video projector 30 can be operative for 25% . In a 
practical system the ratios would possible by 70% and 20% with a "dead" 
changeover period. 

In this case the actor will see the projected plan for a period fifty 
15 times every second which will appear as a continuous display. The 
frequency allows the projected plan to follow the movement of animated 
characters or other 3D objects. 

Once the projected plan has been set up and projected on the floor, 
20 if the 3D virtual objects are stationary then the plan will remain static. 
However, if She 3D virtual object changes shape or moves then it will be 
necessary to change the projection. 

The animation controller 70 has the necessary data to prepare the 
25 3D model. The machine may prepare the model in advance using, for 
example, SOFTIMAGE or Alias 3D model design codes. 

The program interrogates the controller 70 to verify if any changes 
have been made to the 3D model (like motion or shape change (step 804)). 
30 The upper and lower clipping plane to floor height is set according to the 
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presenter's height (steps 806,808). If changes are made then these are 
incorporated at each change (step 810). 

The viewing position for the top view is set (set (812) followed by 
5 setting of the orthogonal projection (step 814). 

All 3 D model elements are changed to form a 2D black and white 
image (step 816) and this is transmitted to all video channels (step 818). 

10 The program is then repeated. 
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CLAIMS 

1. A virtual studio including a virtual studio projection system 
comprising projector means for projecting on to the foreground area floor 
of a virtual studio a projection plan profile of a virtual 3D object. 

2. A virtual studio as claimed in claim 1 in which the projection 
means comprises a video projector situated above the foreground area 
floor. 

3. A virtual studio as claimed in claim 2 in which one or more such 
video projectors may be used to cover a large area. 

4. A virtual studio as claimed in claim 1 in which the foreground area 
may preferably be a chroma-key coloured surface. 

5. A virtual studio as claimed in claim 1 in which the video projector 
may project on to the floor an outline in black and white but still within 
the range of the chroma-keyer. 

6. A video studio as claimed in claim 5 in which a TV camera 
shooting the virtual studio foreground area and the video projector are 
synchronised such that they are not operative together. 

7. A virtual studio as claimed in claim 6 in which the TV camera is 
arranged to be operative for less than three quarters (75%) of a video field 
and the overhead projection camera for less than one quarter (25%). 

8. A virtual studio as claimed in claim 7 in which there is a time delay 
between switching of the video camera and the overhead projection to 
ensure no overlap. 

10 



9. A virtual studio as claimed in any one of the preceding claims in 
which the size of the virtual object at the height of each actor is sampled 
and this is used to produce the projected plan. 

10. A virtual studio including a projection system comprising projector 
means for projecting on to foreground floor area of the studio a projector 
plan profile of a virtual 3D object, the projection system being 
synchronised to electronic shutter means on the camera to project the plan 
profile only during periods when the camera shutter is closed. 



11 




.The 



Application No: 
Claims searched: 



GB 9706192.3 
1 to 10 



Examkaen John Donaldson 

Date of search: 22 May 1998 



Patents Act 1977 

Search Report under Section 17 

Databases searched: 



UK Patent Office collections, including GB, EP, WO & US patent specifications, in: 
UK CI (Ed.P): H4F(FAAX, FCCX, FGJ, FGS, FGT) 

Int CI (Ed.6): H04N 5/00, 5/222, 5/225, 5/247, 5/262, 5/272, 7/00, 9/00, 9/04, 9/64 
9/74, 9/75, 11/00 

Other; Online: WPI 



Documents considered to be relevant: 



Category 


Identity of document and relevant passage 


Relevant 
to claims 


A 


GB 2 1 62628 A (APOGEE) , see abstract 







X Document indicating lack of novelty or inventive step A Document indicating technological background and/or state of the an. 

Y Document indicating lack of inventive step if combined P Document published on or after the declared priority date but before 

with one or more other documents of same category. the filing date of this invention. 

E Patent document published on or after, but with priority date earlier 

& Member of the same patent family than, the filing date of this application. 



An Executive Agency of the Department of Trade and Industry 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



